Bharatanatyam is a traditional Indian dance form that involves the use of facial expressions and body movements to tell stories. A key aspect of Bharatanatyam is the use of hand gestures, also known as hastas, which are used to communicate with specificity and precision. Hastas are symbols, and along with facial expressions and body movements that are contextually relevant, they help to communicate narratives. I am a neuroscientist and have been immersed in Bharatanatyam for 25 years; true to (...) the tradition of the form that emphasizes lifelong scholarship and immersion, I continue to learn from my gurus and supplement my dance training with the study of Carnatic music and Sanskrit. My journey in creative aging started after losing my mother and witnessing the lack of access to expressive movement that was available to her; for fear of falls, my previously-dynamic mother spent the last three months of her life without leaving the bed or feeling the sunshine on her skin. By using hastas in the context of creative aging, I describe how we can promote the acquisition of new skills, the physical benefits even in the face of arthritis and limited mobility, the ability to ascribe meaning to the gestures, and the capability to form new meanings and new gestures that are contemporary and relevant to the lives of older adults. Above all, we can engage older adults actively in the creation and appreciation of art. (shrink)

In order to produce a coherent narrative, speakers must identify the characters in the tale so that listeners can figure out who is doing what to whom. This paper explores whether speakers use gesture, as well as speech, for this purpose. English speakers were shown vignettes of two stories and asked to retell the stories to an experimenter. Their speech and gestures were transcribed and coded for referent identification. A gesture was considered to identify a referent if it was (...) produced in the same location as the previous gesture for that referent. We found that speakers frequently used gesture location to identify referents. Interestingly, however, they used gesture most often to identify referents that were also uniquely specified in speech. Lexical specificity in referential expressions in speech thus appears to go hand‐in‐hand with specification in referential expressions in gesture. (shrink)

During face‐to‐face conversation, transitions between speaker turns are incredibly fast. These fast turn exchanges seem to involve next speakers predicting upcoming semantic information, such that next turn planning can begin before a current turn is complete. Given that face‐to‐face conversation also involves the use of communicative bodily signals, an important question is how bodily signals such as co‐speech hand gestures play into these processes of prediction and fast responding. In this corpus study, we found that hand (...) class='Hi'>gestures that depict or refer to semantic information started before the corresponding information in speech, which held both for the onset of the gesture as a whole, as well as the onset of the stroke (the most meaningful part of the gesture). This early timing potentially allows listeners to use the gestural information to predict the corresponding semantic information to be conveyed in speech. Moreover, we provided further evidence that questions with gestures got faster responses than questions without gestures. However, we found no evidence for the idea that how much a gesture precedes its lexical affiliate (i.e., its predictive potential) relates to how fast responses were given. The findings presented here highlight the importance of the temporal relation between speech and gesture and help to illuminate the potential mechanisms underpinning multimodal language processing during face‐to‐face conversation. (shrink)

The ubiquitous human practice of spontaneously gesturing while speaking demonstrates the embodiment, embeddedness, and sociality of cognition. The present essay takes gestural practice to be a paradigmatic example of a more general claim: human cognition is social insofar as our embedded, intelligent, and interacting bodies select and construct meaning in a way that is intersubjectively constrained and defeasible. Spontaneous co-speech gesture is markedly interesting because it at once confirms embodied aspects of linguistic meaning-making that formalist and linguistic turn-type philosophical approaches (...) fail to appreciate, and it also forefronts intersubjectivity as an inherent and inherently normative dimension of communicative action. Co-speech hand gestures, as linguistically meaningful speech acts, demonstrate both sedimentation and spontaneity (in the sense of Maurice Merleau-Ponty’s dialectic of linguistic expression ( 2002 )), or features of convention and nonconvention in a Gricean sense ( 1989 ). Yet neither pragmatic nor classic phenomenological approaches to communication can accommodate the practice of co-speech hand gesturing without some rehabilitation and reorientation. Pragmatic criteria of intersubjectivity, normativity, and rationality need to confront the non-propositional and nonverbal meaning-making of embodied encounters. Phenomenological treatments of expression and intersubjectivity must consider the normative nature of high-order social practices like language use. Reciprocally critical exchanges between these traditions and gesture studies yield an improved philosophy that treats language as a multi-modal medium for collaborative meaning achievement. The proper paradigm for these discussions is found in enactive approaches to social cognition. Co-speech hand gestures are first and foremost emergent elements of social interaction, not the external whirring of an isolated internal consciousness. In contrast to current literature that frequently presents gestures as uncontrollable bodily upsurge or infallible imagistic phenomenon that drives and dances with verbal or “linguistic” convention (McNeill 1992 , 2005 ), I suggest that we study gestures as dynamic, embodied, and shared tools for collaborative sense-making. (shrink)

A beneficial effect of gesture on learning has been demonstrated in multiple domains, including mathematics, science, and foreign language vocabulary. However, because gesture is known to co-vary with other non-verbal behaviors, including eye gaze and prosody along with face, lip, and body movements, it is possible the beneficial effect of gesture is instead attributable to these other behaviors. We used a computer-generated animated pedagogical agent to control both verbal and non-verbal behavior. Children viewed lessons on mathematical equivalence in which an (...) avatar either gestured or did not gesture, while eye gaze, head position, and lip movements remained identical across gesture conditions. Children who observed the gesturing avatar learned more, and they solved problems more quickly. Moreover, those children who learned were more likely to transfer and generalize their knowledge. These findings provide converging evidence that gesture facilitates math learning, and they reveal the potential for using technology to study non-verbal behavior in controlled experiments. (shrink)

We examine the embodiment of one foundational aspect of human cognition, language, through its bodily association with the gestures that accompany its expression in speech. Gesture is a universal feature of human communication. Gestures are produced by all speakers in every culture . They are tightly timed with speech . Gestures convey important communicative information to the listener, but even blind speakers gesture while talking to blind listeners , so the mutual co-occurrence of speech and gesture reflects (...) a deep association between the two modes that transcends the intentions of the speaker to communicate. Indeed, we believe that this linkage of the vocal expression of language and the arm movements produced with it are a manifestation of the embodiment of thought: that human mental activities arise through bodily interactions with the world and remain linked with them throughout the lifespan. In particular, we propose that speech and gesture have their developmental origins in early hand-mouth linkages, such that as oral activities become gradually used for meaningful speech, these linkages are maintained and strengthened. Both hand and mouth are tightly coupled in the mutual cognitive activity of language. In short, it is the initial sensorimotor linkages of these systems that form the bases for their later cognitive interdependence. (shrink)

This article argues for the gestural origins of speech and language based on the available evidence gathered in humans and nonhuman primates and especially from ape studies. The strong link between motor functions and speech in humans is reviewed. The presence of asymmetrical cerebral organization in nonhuman primates along with functional asymmetries in the perception and production of vocalizations and in intentional referential gestural communication is then emphasized. The nature of primate communicatory systems is presented, and the similarities and differences (...) between these systems and human speech are discussed. It is argued that recent findings concerning neuroanatomical asymmetries in the chimpanzee brain and the existence of both mirror neurons and lateralized use of hands and vocalizations in communication necessitate a reconsideration of the phylogenic emergence of the cerebral and behavioral prerequisites for human speech. (shrink)

This article describes how cc. 39 and 37 create distinct tones of voice and use them to preclude the social pretensions of Egnatius in different spheres. The style of c. 39, markedly oratorical—and non-Catullan—in the syntax of its opening lines, develops into the voice of a respectable senex by way of archaisms of vocabulary and syntax and is capped by a figure of humor otherwise absent from the polymetrics, the apologus. The style thus creates a voice perfectly suited to (...) chastise Egnatius' social ineptitudes and, more importantly, constitutes on the verbal level an embodiment of the standards of the urban elite. Catullus thus illustrates to Egnatius that a subtle system of social gestures can be learned - something which Egnatius, for all his apparent pretensions to social prominence, manifestly has not grasped, marred as he is by the habit of grinning inappropriately. C. 37, which ends with an attack on the same Egnatius, is far different in tone, commingling tokens of artfulness with vulgarity and forcefulness. That mixed style exercises ironic decorum towards Lesbia's lovers, who are themselves an oddly mixed group, well-off on the one hand, but "backstreet adulterers" on the other. But the two tones of voice, and indeed the two groups of lovers, also embody the paradoxes of the Roman construction of stylish behavior, which could be represented as elegant dalliance or reprehensible vice. Inasmuch as Egnatius' fashion affectations, ridiculed at the poem's end, fail to qualify him for the ranks of Lesbia's lovers, he is represented as outside sophisticated society, however it be constructed. In concert the two poems utterly derail Egnatius' social pretensions: c. 39 as it were outflanks him on the "right," barring him from respectable society, and c. 37 outflanks him on the "left," barring him from the "jet set.". (shrink)

Do English speakers think about time the way they talk about it? In spoken English, time appears to flow along the sagittal axis (front/back): the future is ahead and the past is behind us. Here we show that when asked to gesture about past and future events deliberately, English speakers often use the sagittal axis, as language suggests they should. By contrast, when producing co-speech gestures spontaneously, they use the lateral axis (left/right) overwhelmingly more often, gesturing leftward for earlier (...) times and rightward for later times. This left-right mapping of time is consistent with the flow of time on calendars and graphs in English-speaking cultures, but is completely absent from conventional spoken metaphors. English speakers gesture on the lateral axis even when they are using front/back metaphors in their co-occurring speech. This speech-gesture dissociation is not due to any lack of lexical or constructional resources to spatialize time laterally in language, nor to any lack of physical resources to spatialize time sagittally in gesture. We propose that when speakers are describing sequences of events, they often use neither the Moving Ego nor Moving Time perspectives. Rather, they adopt a “Moving Attention” perspective, which is grounded in patterns of interaction with cultural artifacts, not in patterns of interaction with the natural environment. We suggest possible pragmatic, kinematic, and mnemonic motivations for the use of a lateral mental timeline in gesture and in thought. Gestures reveal an implicit spatial conceptualization of time that cannot be inferred from language. (shrink)

A number of language evolution researchers have argued that while language as we now know it is a predominately vocal affair, early language plausibly made extensive use of gesture. Relatedly, these same researchers often claim that while modern language in general uses arbitrary symbols, it is very likely that early language made extensive use of iconicity. Anyone accepting an account of early language along these lines must therefore explain how language shifted over time from a heavily gestural and iconic communication (...) system to a predominately vocal and highly arbitrary one. This article looks at an intriguing theory advanced by Woll as to how this shift was made. We find the spirit of the theory plausible but take issue with its details. Specifically, we identify a serious tension at the core of the theory. We then go on to explore how that tension might be plausibly resolved. The result is a blueprint for a much-improved version of the theory, in our view. (shrink)

There has long been interest in why languages are shaped the way they are, and in the relationship between sign language and gesture. In sign languages, entity classifiers are handshapes that encode how objects move, how they are located relative to one another, and how multiple objects of the same type are distributed in space. Previous studies have shown that hearing adults who are asked to use only manual gestures to describe how objects move in space will use (...) class='Hi'>gestures that bear some similarities to classifiers. We investigated how accurately hearing adults, who had been learning British Sign Language for 1–3 years, produce and comprehend classifiers in locative and distributive constructions. In a production task, learners of BSL knew that they could use their hands to represent objects, but they had difficulty choosing the same, conventionalized, handshapes as native signers. They were, however, highly accurate at encoding location and orientation information. Learners therefore show the same pattern found in sign-naïve gesturers. In contrast, handshape, orientation, and location were comprehended with equal accuracy, and testing a group of sign-naïve adults showed that they too were able to understand classifiers with higher than chance accuracy. We conclude that adult learners of BSL bring their visuo-spatial knowledge and gestural abilities to the tasks of understanding and producing constructions that contain entity classifiers. We speculate that investigating the time course of adult sign language acquisition might shed light on how gesture became conventionalized during the genesis of sign languages. (shrink)

Speakers of many languages prefer allocentric frames of reference (FoRs) when talking about small‐scale space, using words like “east” or “downhill.” Ethnographic work has suggested that this preference is also reflected in how such speakers gesture. Here, we investigate this possibility with a field experiment in Juchitán, Mexico. In Juchitán, a preferentially allocentric language (Isthmus Zapotec) coexists with a preferentially egocentric one (Spanish). Using a novel task, we elicited spontaneous co‐speech gestures about small‐scale motion events (e.g., toppling blocks) in (...) Zapotec‐dominant speakers and in balanced Zapotec‐Spanish bilinguals. Consistent with prior claims, speakers’ spontaneous gestures reliably reflected either an egocentric or allocentric FoR. The use of the egocentric FoR was predicted—not by speakers’ dominant language or the language they used in the task—but by mastery of words for “right” and “left,” as well as by properties of the event they were describing. Additionally, use of the egocentric FoR in gesture predicted its use in a separate nonlinguistic memory task, suggesting a cohesive cognitive style. Our results show that the use of spatial FoRs in gesture is pervasive, systematic, and shaped by several factors. Spatial gestures, like other forms of spatial conceptualization, are thus best understood within broader ecologies of communication and cognition. (shrink)

Speakers perform manual gestures in the physical space nearest them, called gesture space. We used a controlled elicitation task to explore whether speakers use gesture space in a consistent way and whether they use space in a contrastive way when talking about abstract referents. Participants answered two questions designed to elicit contrastive, abstract discourse. We investigated manual gesture behavior. Gesture hand, location on the horizontal axis, and referent in corresponding speech were coded. We also coded contrast in speech. (...) Participants’ overall tendency to use the same hand = 13.12, p =.001, 95% CI [.31,.43], d = 2.53) and same location = 7.47, p =.001, 95% CI [.27,.47], d = 1.69) when referring to an entity was higher than expected frequency. When comparing pairs of gestures produced with contrastive speech to pairs of gestures produced with non-contrastive speech, we found a greater tendency to produce gestures with different hands for contrastive speech: = 4.19, p =.001, 95% CI [.27,.82], d = 1.42). We did not find associations between dominant side and positive concepts or between left, center, and right space and past, present, and future, respectively, as predicted by previous studies. Taken together, our findings suggest that speakers do produce spatially consistent and contrastive gestures for abstract as well as concrete referents. They may be using spatial resources to assist with abstract thinking, and/or to help interlocutors with reference tracking. Our findings also highlight the complexity of predicting gesture hand and location, which appears to be the outcome of many competing variables. (shrink)

Visual information conveyed by iconic hand gestures and visible speech can enhance speech comprehension under adverse listening conditions for both native and non‐native listeners. However, how a listener allocates visual attention to these articulators during speech comprehension is unknown. We used eye‐tracking to investigate whether and how native and highly proficient non‐native listeners of Dutch allocated overt eye gaze to visible speech and gestures during clear and degraded speech comprehension. Participants watched video clips of an actress uttering (...) a clear or degraded (6‐band noise‐vocoded) action verb while performing a gesture or not, and were asked to indicate the word they heard in a cued‐recall task. Gestural enhancement was the largest (i.e., a relative reduction in reaction time cost) when speech was degraded for all listeners, but it was stronger for native listeners. Both native and non‐native listeners mostly gazed at the face during comprehension, but non‐native listeners gazed more often at gestures than native listeners. However, only native but not non‐native listeners' gaze allocation to gestures predicted gestural benefit during degraded speech comprehension. We conclude that non‐native listeners might gaze at gesture more as it might be more challenging for non‐native listeners to resolve the degraded auditory cues and couple those cues to phonological information that is conveyed by visible speech. This diminished phonological knowledge might hinder the use of semantic information that is conveyed by gestures for non‐native compared to native listeners. Our results demonstrate that the degree of language experience impacts overt visual attention to visual articulators, resulting in different visual benefits for native versus non‐native listeners. (shrink)

Gestures’ Contribution to Collective Metaphorical Thinking in a Community of Philosophical Inquiry (CPI). This paper explores an idea expressed by a student discussing where our thoughts come from: to think we have to move our hands. Such sentence echoes the literature on the role of gesture for thinking. This study also focuses on the collective advancement of reasoning in a CPI. The instructor chooses to conclude by asking each student to suggest an analogy of thinking. This closing sequence reveals (...) how the instructor, through metaphorical gestures, fosters collective awareness of new propositions, and their further elaboration. After characterizing the cognitive models so produced, video analysis is used to follow their collective, verbal and gestural construction along the discussion. (shrink)

In this article I argue that, with the liberation of the hand from the tasks of locomotion in human evolution, unconscious use of the hands begins to create cultural forms. The first feature of the hands is its openness to the world. The second feature is its mediation between things and the body of which it is a part. The third feature is its self-referentiality. By touching, by giving form to a material, by gestures, and by establishing symbolic (...) order in the given world, the actions of the hand develop important preconditions for language, aesthetic forms, and social order. The hand itself assumes the most diverse shapes and gives different shapes to the objects around it. Touching and pointing produce the certainty that the indicated object exists, insofar as it is produced as object-for-the-hand. Its doubling in the use of the hand re-creates the object in a symbolic medium. This mimetic re-creation of the world is the main characteristic of the creativity of hand use. The two-sided plasticity of the hands divides the social universe into right and left halves, classified according to right-handed and left-handed values. The hand’s actual acquisition of this capacity is the result of a social process. In coordination with our visual sense, the use of the hands leads both to the making of objects and to the construction of worlds of subjective feelings. Both achievements are often ascribed to language. But the functions considered as most fundamental to language first take shape in the unconscious use of our hands. (shrink)

The gestural repertoire of captive gorillas contains the so-called “hand-on“ (or “pat-off“) gesture in which one animals puts its flat hand on top of another's head, which often leads to cessation of the receiver's previous activity. We investigate the origins of this gesture and developmental aspects of gesture creation. We further analyze gesture form and use in relation to the age of the sender with special consideration of the reaction of the receiver to better explain the function of (...) the hand-on. The focus of the study is to compare this gesture across different age groups in gorillas in regard to both production and reception and to get insights into possible origins of the gesture. (shrink)

Brain–machine interfaces are systems that allow the control of a device such as a robot arm through a person’s brain activity; such devices can be used by disabled persons to enhance their life and improve their independence. This paper is an extended version of a work that aims at discriminating between left and right imagined hand movements using a support vector machine classifier to control a robot arm in order to help a person to find an object in the (...) environment. The main focus here is to search for the best features that describe efficiently the electroencephalogram data during such imagined gestures by comparing two feature extraction methods, namely the continuous wavelet transform and the empirical modal decomposition, combined with the principal component analysis that were fed through a linear and radial basis function kernel SVM classifier. The experimental results showed high performance achieving an average accuracy across all the subjects of 92.75% with an RBF kernel SVM classifier using CWT and PCA compared to 80.25% accuracy obtained with EMD and PCA. The proposed system has been implemented and tested using data collected from five male subjects and it enabled the control of the robot arm in the right and the left direction. (shrink)

Right-hand preference for conversational gestures does not imply close connections between the neural systems controlling manual and vocal communication. Use of speech and gestures may dissociate in some cases of focal brain damages. Furthermore, there are limits in the ability to combine spoken words and concurrent hand movements. These findings suggest that discourse production depends on multiple components which probably have different evolutionary origins.

Successful face‐to‐face communication involves multiple channels, notably hand gestures in addition to speech for spoken language, and mouth patterns in addition to manual signs for sign language. In four experiments, we assess the extent to which comprehenders of British Sign Language (BSL) and English rely, respectively, on cues from the hands and the mouth in accessing meaning. We created congruent and incongruent combinations of BSL manual signs and mouthings and English speech and gesture by video manipulation and asked (...) participants to carry out a picture‐matching task. When participants were instructed to pay attention only to the primary channel, incongruent “secondary” cues still affected performance, showing that these are reliably used for comprehension. When both cues were relevant, the languages diverged: Hand gestures continued to be used in English, but mouth movements did not in BSL. Moreover, non‐fluent speakers and signers varied in the use of these cues: Gestures were found to be more important for non‐native than native speakers; mouth movements were found to be less important for non‐fluent signers. We discuss the results in terms of the information provided by different communicative channels, which combine to provide meaningful information. (shrink)

Gestures—hand movements that accompany speech and express ideas—can help children learn how to solve problems, flexibly generalize learning to novel problem‐solving contexts, and retain what they have learned. But does it matter who is doing the gesturing? We know that producing gesture leads to better comprehension of a message than watching someone else produce gesture. But we do not know how producing versus observing gesture impacts deeper learning outcomes such as generalization and retention across time. Moreover, not all (...) children benefit equally from gesture instruction, suggesting that there are individual differences that may play a role in who learns from gesture. Here, we consider two factors that might impact whether gesture leads to learning, generalization, and retention after mathematical instruction: (1) whether children see gesture or do gesture and (2) whether a child spontaneously gestures before instruction when explaining their problem‐solving reasoning. For children who spontaneously gestured before instruction, both doing and seeing gesture led to better generalization and retention of the knowledge gained than a comparison manipulative action. For children who did not spontaneously gesture before instruction, doing gesture was less effective than the comparison action for learning, generalization, and retention. Importantly, this learning deficit was specific to gesture, as these children did benefit from doing the comparison manipulative action. Our findings are the first evidence that a child's use of a particular representational format for communication (gesture) directly predicts that child's propensity to learn from using the same representational format. (shrink)

When young children learn to use language, they start to use their hands in co‐verbal gesturing. There are, however, considerable differences between children, and it is not completely understood what these individual differences are due to. We studied how children at 4 years of age employ speech and iconic gestures to convey meaning in different kinds of spatial event descriptions, and how this relates to their cognitive abilities. Focusing on spontaneous illustrations of actions, we applied a semantic feature (SF) (...) analysis to characterize combinations of speech and gesture meaning and related them to the child's visual‐spatial abilities or abstract/concrete reasoning abilities (measured using the standardized SON‐R test). Results show that children with higher cognitive abilities convey significantly more meaning via gesture and less solely via speech. These findings suggest that young children's use of cospeech representational gesturing is positively related to their mental representation and reasoning abilities. (shrink)

For humans, the ability to communicate and use language is instantiated not only in the vocal modality but also in the visual modality. The main examples of this are sign languages and gestures. Sign languages, the natural languages of Deaf communities, use systematic and conventionalized movements of the hands, face, and body for linguistic expression. Co-speech gestures, though non-linguistic, are produced in tight semantic and temporal integration with speech and constitute an integral part of language together with speech. (...) The articles in this issue explore and document how gestures and sign languages are similar or different and how communicative expression in the visual modality can change from being gestural to grammatical in nature through processes of conventionalization. As such, this issue contributes to our understanding of how the visual modality shapes language and the emergence of linguistic structure in newly developing systems. Studying the relationship between signs and gestures provides a new window onto the human ability to recruit multiple levels of representation in the service of using or creating conventionalized communicative systems. (shrink)

Human languages typically employ a variety of spatial metaphors for time (e.g., “I'm looking forward to the weekend”). The metaphorical grounding of time in space is also evident in gesture. The gestures that are performed when talking about time bolster the view that people sometimes think about regions of time as if they were locations in space. However, almost nothing is known about the development of metaphorical gestures for time, despite keen interest in the origins of space–time metaphors. (...) In this study, we examined the gestures that English‐speaking 6‐to‐7‐year‐olds, 9‐to‐11‐year‐olds, 13‐to‐15‐year‐olds, and adults produced when talking about time. Participants were asked to explain the difference between pairs of temporal adverbs (e.g., “tomorrow” versus “yesterday”) and to use their hands while doing so. There was a gradual increase across age groups in the propensity to produce spatial metaphorical gestures when talking about time. However, even a substantial majority of 6‐to‐7‐year‐old children produced a spatial gesture on at least one occasion. Overall, participants produced fewer gestures in the sagittal (front‐back) axis than in the lateral (left‐right) axis, and this was particularly true for the youngest children and adolescents. Gestures that were incongruent with the prevailing norms of space–time mappings among English speakers (leftward and backward for past; rightward and forward for future) gradually decreased with increasing age. This was true for both the lateral and sagittal axis. This study highlights the importance of metaphoricity in children's understanding of time. It also suggests that, by 6 to 7 years of age, culturally determined representations of time have a strong influence on children's spatial metaphorical gestures. (shrink)

Speakers often use gesture to demonstrate how to perform actions—for example, they might show how to open the top of a jar by making a twisting motion above the jar. Yet it is unclear whether listeners learn as much from seeing such gestures as they learn from seeing actions that physically change the position of objects (i.e., actually opening the jar). Here, we examined participants' implicit and explicit understanding about a series of movements that demonstrated how to move a (...) set of objects. The movements were either shown with actions that physically relocated each object or with gestures that represented the relocation without touching the objects. Further, the end location that was indicated for each object covaried with whether the object was grasped with one or two hands. We found that memory for the end location of each object was better after seeing the physical relocation of the objects, that is, after seeing action, than after seeing gesture, regardless of whether speech was absent (Experiment 1) or present (Experiment 2). However, gesture and action built similar implicit understanding of how a particular handgrasp corresponded with a particular end location. Although gestures miss the benefit of showing the end state of objects that have been acted upon, the data show that gestures are as good as action in building knowledge of how to perform an action. (shrink)

The steady stream of books on medieval manuscripts addressed to a popular audience over the past two decades coincides with the advent of tablets such as Amazon's Kindle. As the flatlands of the digital realm encompass more of life, nostalgia for a tactile realm of reading, whether in the making or the perception of artifacts, asserts itself, as does the desire to immerse oneself in the real space of the conventional book, as opposed to the virtual yet denatured spaces of (...) the metaverse. No accident, then, that, following an introduction that gestures toward medieval subjects and to readers other than Christian, Wellesley's engaging, if episodic, account of medieval manuscripts (published in the United States by Basic Books as The Gilded Page: The Secret Lives of Medieval Manuscripts) opens with a prologue in which she recounts her hands-on visit to a parchment maker, during which she tried to slough the hair off a hide using an implement known as a scudder.Manuscripts permit one to come into contact with the past, or at least to harbor the illusion that one can. Wellesley's book repeatedly evokes the contingency of landmarks and masterworks that make up the history of culture. Pendant chapters treat “Discoveries,” medieval as well as modern, of books that, when first encountered, had been lost to view for the better part of a millennium, as well as the “Near-Disasters” by which famous works such as Beowulf (or indeed entire libraries) made it down to modernity by only a hair's breadth—despite the devil's assurance in Bulgakov's The Master and Margarita that “manuscripts don't burn.” Within a binding held tight by clasps, parchment—even when exposed to fire—can survive, if it is not then reduced to a mess by the water used to douse the flames. Having been rescued from the ruins of Ashburnham House, which burned in 1731, the detached pages of the Cotton Genesis at the British Library—one of a handful of illustrated biblical manuscripts from late antiquity—now look like soggy, burned toast pressed between glass sheets to conserve what little is left.Other chapters in this book make things personal, matching brief biographies of celebrated manuscripts with famous (or not so famous) persons, whether patrons, scribes, or artists—a sequence that reflects their order of importance in the Middle Ages. To have included an afterword as well as an epilogue seems indulgent: one, too brief, on the impact of printing (focused on the English printer Caxton), the other on “The Use and Misuse of the Past,” which dwells on the dispersal of monastic libraries after the English Reformation. The focus on the British Isles, to the neglect of the continent, risks being too narrow for a book so general in scope. The book's very British flavor, however, is also part of what lends it a familiar, chatty tone. A pleasant, companionable read, this extended essay does not advance our knowledge of medieval manuscripts, but it does help diffuse it to a larger audience in a way that makes old books seem companionable. Rather than Hidden Hands, the book might well have been called Comites latentes: hidden friends. (shrink)

Studies of the evolution of language rely heavily on comparisons to nonhuman primates, particularly the gestural communication of nonhuman apes. Differences between human and ape gestures are largely ones of degree rather than kind. For example, while human gestures are more flexible, ape gestures are not inflexible. In this piece, I closely consider two features of the gestural communication of apes and humans that might display differences in kind: iconicity and temporal alignment. Iconicity has long played a (...) privileged role in theories of gestural language origins, the proposal being that it provided a steppingstone into language through a stage of pantomime. However, iconicity is not as easy as it seems. Evidence from co-speech gestures of hearing children and from homesign suggests that iconic reference is both cognitively complex and slow to develop in humans. There is no conclusive evidence that nonhuman apes understand or produce iconic gestures; some gestures may appear iconic to human observers but the apes themselves may not understand the similarity between form and meaning. Far from providing an easy pathway into the emergence of symbolic communication, iconicity relies on sophisticated capacities for analogy and abstraction, ones often lacking in apes and young children. Temporal alignment between gesture and vocalization is another area that may show sharp contrasts between adult humans and apes, though data here is sparser. I discuss the tight synchronization of gesture and speech that emerges over the first year of life in human children and contrast it with gesture and vocalization in apes, which typically are described as overlapping but not simultaneous. Human ancestors probably communicated in ways similar to other apes, but the dual emergence of the ability to use iconic reference and the alignment of hand and mouth may have set our ancestors down a unique evolutionary road toward language. (shrink)

Current conceptions of human language include a gestural component in the communicative event. However, determining how the linguistic and gestural signals are distinguished, how each is structured, and how they interact still poses a challenge for the construction of a comprehensive model of language. This study attempts to advance our understanding of these issues with evidence from sign language. The study adopts McNeill's criteria for distinguishing gestures from the linguistically organized signal, and provides a brief description of the linguistic (...) organization of sign languages. Focusing on the subcategory of iconic gestures, the paper shows that signers create iconic gestures with the mouth, an articulator that acts symbiotically with the hands to complement the linguistic description of objects and events. A new distinction between the mimetic replica and the iconic symbol accounts for the nature and distribution of iconic mouth gestures and distinguishes them from mimetic uses of the mouth. Symbiotic symbolization by hand and mouth is a salient feature of human language, regardless of whether the primary linguistic modality is oral or manual. Speakers gesture with their hands, and signers gesture with their mouths. (shrink)

The role of nonverbal communication in patients with post-stroke language impairment (aphasia) is not yet fully understood. This study investigated how aphasic patients perceive and produce co-speech gestures during face-to-face interaction, and whether distinct brain lesions would predict the frequency of spontaneous co-speech gesturing. For this purpose, we recorded samples of conversations in patients with aphasia and healthy participants. Gesture perception was assessed by means of a head-mounted eye-tracking system, and the produced co-speech gestures were coded according to (...) a linguistic classification system. The main results are that meaning-laden gestures (e.g., iconic gestures representing object shapes) are more likely to attract visual attention than meaningless hand movements, and that patients with aphasia are more likely to fixate co-speech gestures overall than healthy participants. This implies that patients with aphasia may benefit from the multimodal information provided by co-speech gestures. On the level of co-speech gesture production, we found that patients with damage to the anterior part of the arcuate fasciculus showed a higher frequency of meaning-laden gestures. This area lies in close vicinity to the premotor cortex and is considered to be important for speech production. This may suggest that the use of meaning-laden gestures depends on the integrity of patients’ speech production abilities. (shrink)

Public speakers like politicians carefully craft their words to maximize the clarity, impact, and persuasiveness of their messages. However, these messages can be shaped by more than words. Gestures play an important role in how spoken arguments are perceived, conceptualized, and remembered by audiences. Studies of political speech have explored the ways spoken arguments are used to persuade audiences and cue applause. Studies of politicians’ gestures have explored the ways politicians illustrate different concepts with their hands, but have (...) not focused on gesture's potential as a tool of persuasion. Our paper combines these traditions to ask first, how politicians gesture when using spoken rhetorical devices aimed at persuading audiences, and second, whether these gestures influence the ways their arguments are perceived. Study 1 examined two rhetorical devices—contrasts and lists—used by three politicians during U.S. presidential debates and asked whether the gestures produced during contrasts and lists differ. Gestures produced during contrasts were more likely to involve changes in hand location, and gestures produced during lists were more likely to involve changes in trajectory. Study 2 used footage from the same debates in an experiment to ask whether gesture influenced the way people perceived the politicians’ arguments. When participants had access to gestural information, they perceived contrasted items as more different from one another and listed items as more similar to one another than they did when they only had access to speech. This was true even when participants had access to only gesture (in muted videos). We conclude that gesture is effective at communicating concepts of similarity and difference and that politicians (and likely other speakers) take advantage of gesture's persuasive potential. (shrink)

The technology, which is implemented with cross platform tools, is proposed for modeling of gesture units of sign language, animation between states of gesture units with a combination of gestures. Implemented technology simulates sequence of gestures using virtual spatial hand model and performs recognition of dactyl items from camera input using trained on collected training dataset set convolutional neural network, based on the MobileNetv3 architecture, and with the optimal configuration of layers and network parameters. On the collected (...) test dataset accuracy of over 98% is achieved. (shrink)

A leading philosopher and theologian, Jean-Louis Chrétien uses poetry and painting to explore a theme that runs through all of his work: how human life is shaped by the experience of call and response. For Chrétien, we live by responding to the call of experience with words, gestures, expressions, and silence. In luminous meditations on Rembrandt, Delacroix, Manet, Verlaine, Keats, and other artists, Chrétien shows how “talking hands of painters” and the “secretly lucid” voices of poets confront the finitude (...) of the human body. Hand to Hand is a deeply cultured renewal of art in all its provocative, transforming, spiritual presence. (shrink)

Spatial skills predict important life outcomes, such as mathematical achievement or entrance into Science, Technology, Engineering, and Mathematics disciplines. Children significantly vary in their spatial performance even before they enter formal schooling. One correlate of children's spatial performance is the spatial language they produce and hear from others, such as their parents. Because the emphasis has been on spatial language, less is known about the role of hand gestures in children's spatial development. Some children are more likely to (...) fall behind in their spatial skills than others. Children born premature constitute such a risk group. Here, we compared performance of term and preterm-born children on two non-verbal spatial tasks—mental transformation and block design. We also examined relations of children's performance on these tasks to parental spatial language and gesture input and their own production of spatial language and gesture during an independent puzzle play interaction. We found that while term and preterm-born children as a group did not differ in the mental transformation or block design performance, children varied widely in their performance within each group. The variability in mental transformation scores was predicted by both a subset of spatial words and all spatial gestures children produced. Children's spatial language and gesture were in turn related to their parents' spatial language and gesture. Parental spatial language and gesture had an indirect relation on children's mental transformation, but not block design, scores via children's spatial language, and gesture use. Overall, results highlight the unique contributions of speech and gesture in communicating spatial information and predicting children's spatial performance. (shrink)

The aim of this study is to analyze the relationship between rhythmic movements and deictic gestures at the end of the first year of life, and to focus on their unimodal or multimodal character. We hypothesize that multimodal rhythmic movement performed with an object in the hand can facilitate the transition to the first deictic gestures. Twenty-three children were observed at 9 and 12 months of age in a naturalistic play situation with their mother or father. Results (...) showed that rhythmic movements with objects in the hand are a frequent behavior in children's repertoires. Rhythmic behaviors tend to decrease from 9 to 12 months, specifically when they are unimodal. Multimodal rhythmic behavior production at 9 months is positively related with proximal deictic gestures 3 months later. Multimodal rhythmic movements are not directly related to distal deictic gestures, but are indirectly related via proximal deictic gestures. These results highlight the relevance of multimodal behaviors in the transition to the use of early gestures, and can be considered as a transitional phenomenon between the instrumental action and early communicative gestures. (shrink)

For Greek antiquity, the question of right or fitting measure constituted the very heart of both ethics and politics. But can the Good of the ethical life and the Justice of the political be reduced to measurement and calculation? If they are matters of measure, are they not also absolutely immeasurable? In critical dialogue with texts by Plato, Hölderlin, Rilke, Heidegger, Benjamin, Adorno, Marx, Merleau-Ponty, Levinas, and Levi, the author argues that the question of measure has become ever more urgent (...) in the context of a modernity pressured by the conditions of a technological economy and a relativism that threatens to destroy a vital sense of moral responsibility and the commitment to justice that underlies the possibility of freedom. Conceived as a task for the “metaphysics” of memory, this book explores the normative problematic of measure, bringing its deeply buried redemptive promise to appearance in our gestures, uses and abuses of the hands, the dialectic of tact, and the manners of social existence. (shrink)

Using a cross-modal semantic priming paradigm, both experiments of the present study investigated the link between the mental representations of iconic gestures and words. Two groups of the participants performed a primed lexical decision task where they had to discriminate between visually presented words and nonwords (e.g., flirp). Word targets (e.g., bird) were preceded by video clips depicting either semantically related (e.g., pair of hands flapping) or semantically unrelated (e.g., drawing a square with both hands) gestures. The duration (...) of gestures was on average 3,500 ms in Experiment 1 but only 1,000 ms in Experiment 2. Significant priming effects were observed in both experiments, with faster response latencies for related gesture–word pairs than unrelated pairs. These results are consistent with the idea of interactions between the gestural and lexical representational systems, such that mere exposure to iconic gestures facilitates the recognition of semantically related words. (shrink)

This study measured hand and head action strengths of eight typical emotional states using an authentic but implicit emotion elicitation task. Participants listened to and then retold five stories in which eight typical emotional states were experienced by the narrators. The number of hand and head gestures that occur naturally while experiencing an emotional state was used as an index to determine the hand and head action strength of that emotional state. Results showed a larger number (...) of head gestures than hand gestures, suggesting that head action strengths of the eight emotional states are stronger than their hand action strengths. These findings are consistent with the data extracted from Lancaster Sensorimotor Norms (LSN), although the two sets of data were gathered in two completely different experimental conditions and in two different languages. Furthermore, our data showed a prototypical directionality effect for the typical emotional states, specifically, happiness, anger and pride were primarily accompanied by upward gestures but downward gestures for sadness and shame; surprise was primarily accompanied by forward gestures but backward gestures for fear and disgust. (shrink)

Interaction systems with gesture-based touchless inputs are becoming more common. Nevertheless, perceptual properties of the visual feedback used in the system have not been well documented. We investigated whether the speed of motion shown in visual feedback used in gesture-based touchless inputs could be a cue for the heaviness sensation of an object even when other incidental cues, such as changes in object size and spatial consistencies in direction between gestures and feedback, were eliminated from the stimuli. Participants were (...) asked to make a gesture to grasp and raise/lower disks shown on a horizontal display. The disk’s diameter changed in accordance with the vertical position of the participant’s hand. The results showed that the rate of change in diameter determined the heaviness sensation. When the disks were replaced with concentric gratings having sinusoidal radial intensity and thus the cue of size change was eliminated from the stimuli, the heaviness sensation was dependent on the speed of phase shift in the grating. It was also found that spatial consistency between the direction of gestures and phase shift was not a critical condition for the heaviness sensation. Finally, the speed of motion served as a critical determinant of the heaviness sensation even when another visual feature was modulated in a single session, which indicates that the effect of the speed of motion on the heaviness sensation was unlikely due to demanded characteristics. The results indicate that the heaviness sensation for visual feedback of gesture-based touchless inputs is based purely on the speed of the visual feedback motion. (shrink)

Human communication is multi-modal. It is an empirical fact that many of our acts of communication exploit a variety of means to make our communicative intentions recognisable. Scholars readily distinguish between verbal and non-verbal means of communication, and very often they deal with them separately. So it is that a great number of semanticists and pragmaticists give verbal communication preferential treatment. The non-verbal aspects of an act of communication are treated as if they were not underlain by communicative intentions. They (...) are “relegated” to mere aspects of the context. However, several schools of thoughts have a different take on the issue. Thus psychologists or semioticians of gesture have shown how intricately gestures and speech are related in utterances. And, in a different area of the theoretical landscape, so-called “Relevance Theorists” have made the same point. Thus, Robyn Carston writes that “the domain of pragmatics is a natural class of environmental phenomena, that of ostensive stimuli; verbal utterances are the central case, but not the only one, and they themselves are frequently accompanied by other ostensive gestures of the face, hands, voice etc, all of which have to be interpreted together if one is to correctly infer what is being communicated”. This position rests on the assumption that there is a single “pragmatic system” or module at work in the interpretation of “ostensive stimuli”. When it comes to interpreting verbal stimuli, the same mechanisms and resources are used as when it comes to processing non-verbal ones. If there is no distinct “linguistic pragmatic system”, then the scholar who studies communication should not favour the verbal at the expense of the non-verbal. In this paper, I want to make a contribution to the study of multi-modal messages by considering a type of utterances that mix the verbal with the non-verbal in such a way that a piece of non-linguistic communication seems to stand in for a linguistic constituent which remains unrealised. Here is a real-life example : I didn't see the [IMITATION OF FRIGHTENING GRUMPINESS] woman today; will she be back this week? The square-bracketed string in small capitals is meant to capture the facial expressions and gestures performed in the conversational setting. What is intriguing here is that this instance of ostensive mimicry does not come as a mere complement to some linguistic stimulus; it appears to take the place of that stimulus. I shall try to show that a linguistic analysis can indeed be offered for cases like – though, I believe, without succumbing to the pro-linguistic bias that Carston warns against. I will, however, argue against an ellipsis-based account : the structure of sentence does not contain an unrealised adjective phrase. Instead, I shall defend a ‘syntactic-recruitment' account. (shrink)

This essay argues that Merleau-Ponty’s phenomenological description of gestural motility in his Phenomenology of Perception contributes to, and in a material way carries forward, not only (1) the account of alienation that Marx proposes in his writings on the condition of manual labour, but also (2) the reflections, at once critical and utopian, that Marx set out in his 1844 Economic and Philosophical Manuscripts , evoking in terms of praxis the realization and fulfillment of our sensuous nature as embodied beings. (...) It is also argued, here, that Merleau-Ponty’s critique of the philosophical accounts of gesture formulated by intellectualism and naturalism shows that, in spite of their differences, they are both ideologically constituted systems of thought that uncritically reflect, and consequently reproduce, the alienations and reifications to which Marx calls attention in his critique of capitalism. Thus, as long as the invisible hand of capital rules, the hands of the worker that exploitation has damaged - and indeed the fully realized hands of which Marx in his early years dreamed - will remain invisible. Key Words: alienated labour • Marx • Merleau-Ponty • motor pathologies • phenomenology • reification. (shrink)

In the wake of the digital, some have recommended that we abandon the tedium of teaching handwriting to children in service of promoting “more creative” digital literacies. Others worry that an early diet of keyboard and screen may have deleterious effects on children's social, emotional, and cognitive development, as well as their physical well-being. Yet in this debate, the algorithmic scripts and digital surfaces underwriting these new reading, writing, and mathematical practices are, with a few notable exceptions, almost exclusively ignored. (...) In this essay, Catherine Adams asks whether the digital, and the reading and writing spaces it affords, are of consequence to our habits of thinking and ways of being, particularly in light of the possible obsolescence of pen and paper in schools. She shows that writing using a word processor is no mere mechanical pressing of keys, but an intricate ballet of writerly reading eyes and readerly writing hands, caught up in a dynamic environment of algorithmic paratexts and copy-cut-paste thinking. (shrink)

Children learn and use various strategies to solve math problems. One way children's math learning can be supported is through their use of and exposure to hand gestures. Children's self-produced gestures can reveal unique, math-relevant knowledge that is not contained in their speech. Additionally, these gestures can assist with their math learning and problem solving by supporting their cognitive processes, such as executive function. The gestures that children observe during math instructions are also linked to (...) supporting cognition. Specifically, children are better able to learn, retain, and generalize knowledge about math when that information is presented within the gestures that accompany an instructor's speech. To date, no conceptual model provides an outline regarding how these gestures and the math environment are connected, nor how they may interact with children's underlying cognitive capacities such as their executive function. In this review, we propose a new model based on an integration of the information processing approach and theory of embodied cognition. We provide an in-depth review of the related literature and consider how prior research aligns with each link within the proposed model. Finally, we discuss the utility of the proposed model as it pertains to future research endeavors. (shrink)

In the recent years, gesture recognition based on the surface electromyography (sEMG) signals has been extensively studied. However, the accuracy and stability of gesture recognition through traditional machine learning algorithms are still insufficient to some actual application scenarios. To enhance this situation, this paper proposed a method combining feature selection and ensemble extreme learning machine (EELM) to improve the recognition performance based on sEMG signals. First, the input sEMG signals are preprocessed and 16 features are then extracted from each channel. (...) Next, features that mostly contribute to the gesture recognition are selected from the extracted features using the recursive feature elimination (RFE) algorithm. Then, several independent ELM base classifiers are established using the selected features. Finally, the recognition results are determined by integrating the results obtained by ELM base classifiers using the majority voting method. The Ninapro DB5 dataset containing 52 different hand movements captured from 10 able-bodied subjects was used to evaluate the performance of the proposed method. The results showed that the proposed method could perform the best (overall average accuracy 77.9%) compared with decision tree (DT), ELM, and random forest (RF) methods. (shrink)

Past research has suggested that those spontaneous movements of the human hand made during talk convey significant semantic information over and above the speech, at least when the unit of speech analyzed is the individual clause. However, no previous research has tested whether this information is represented linguistically elsewhere in the narrative . The first study, reported here, uses an experimental procedure to identify which specific imagistic gestures add semantic information to the speech. The second study analyzes whether (...) the specific gestures still do this when respondents hear the whole narrative. It was found that two thirds of the semantic information, thought to be carried by the gestures, is, in fact, represented in the linguistic discourse, or is inferable from it. However, one third of the additional semantic information contained in the gestures is not represented linguistically in the narrative nor is it inferable from it. In other words, a proportion of the imagistic gestures that accompany speech are absolutely critical to semantic communication. (shrink)

Silent gesture is not considered to be linguistic, on par with spoken and sign languages. It is claimed that silent gestures, unlike language, represent events holistically, without compositional structure. However, recent research has demonstrated that gesturers use consistent strategies when representing objects and events, and that there are behavioral and clinically relevant limits on what form a gesture may take to effect a particular meaning. This systematicity challenges a holistic interpretation of silent gesture, which predicts that there should be (...) no stable form‐meaning correspondence across event representations. Here, we demonstrate to the contrary that untrained gesturers systematically manipulate the form of their gestures when representing events with and without a theme (e.g., Someone popped the balloon vs. Someone walked), that is, transitive and intransitive events. We elicited silent gestures and annotated them for manual features active in coding transitivity distinctions in sign languages. We trained linear support vector machines to make item‐by‐item transitivity predictions based on these features. Prediction accuracy was good across the entire dataset, thus demonstrating that systematicity in silent gesture can be explained with recourse to subunits. We argue that handshape features are constructs co‐opted from cognitive systems subserving manual action production and comprehension for communicative purposes, which may integrate into the linguistic system of emerging sign languages. We further suggest that nonsigners tend to map event participants to each hand, a strategy found across genetically and geographically distinct sign languages, suggesting the strategy's cognitive foundation. (shrink)

When interacting with others, we often use bodily signals to communicate. Among these signals, pointing, whether with the eyes or the hands, allows coordinating our attention with others, and the perception of pointing gestures implicates a range of social cognitive processes. Here, we review the brain mechanisms underpinning the perception and understanding of pointing, focusing on eye gaze perception and associated joint attention processes. We consider pointing gesture perception, but leave aside pointing gesture execution as it relates to a (...) distinct area of cognitive neuroscience research. We describe the attention orienting effects of pointing and the neural substrates for the perception of biological cues. We consider the multiple high-level social cognitive processes elicited by pointing gesture perception and examine how pointing gestures are related to the general taxonomy of gestures. We conclude by emphasizing that pointing is a social phenomenon and that a full account of pointing will require an integrative approach taking into account the distinct perspectives from which this phenomenon can be investigated. (shrink)